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ABSTRACT 


In  this  paper  we  extend  our  grasp  of  statistical  theory,  duality 
theory,  and  computational  convenience  to  the  general  linear  inequality 

p 

situation  in  K  L  (or  minimum  discrimination  information)  estimation  by 
exhibiting  it  as  the  limit  of  a  simple  one  parameter  sequence  of  equality 
problems.  Only  the  finite  discrete  distribution  case  is  treated  here. 
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INTRODUCTION 


Frequently  in  hypothesis  testing  or  estimation  of  an  "inferential" 
distribution  (in  Akaike's  terminology)  by  the  Khin&in-Kullback-Leibler  (or 
Minimum  Discrimination  Information)  method  one  has  information  about  the 
possible  candidate  distributions  in  the  form  of  linear  inequalities  on 
the  components  of  the  distribution  in  addition  to  equality  (moment)  con¬ 
straints.  In  MOFS  1978,  Chames,  Cooper,  and  Seiford  [1]  developed  a  convex 
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programming  duality  theory  for  the  K  L  method  with  linear  inequality 
constraints.  It  is  especially  incisive  and  convenient  for  constraints  in 
equality  (or  moment)  form  which,  further,  have  connections  with  established 
statistical  theory  as  well  as  the  analytic  and  computational  facility  of 
an  unconstrained  extremal  problem  in  simple  smooth  functions. 

In  this  paper  we  extend  our  grasp  of  statistical  theory,  duality 
theory,  and  computational  convenience  to  the  general  linear  inequality 
situation  by  exhibiting  it  as  the  limit  of  a  simple  one  parameter  sequence 
of  equality  problems.  We  treat  only  the  finite  discrete  distribution  case 
here,  reserving  the  general  distribution  case  to  a  forthcoming  paper  which 
also  simplifies  the  duality  theory  of  the  "equality"  case  for  general 
distributions  as  developed  by  Ben-Tal  and  Charnes  in  [2). 

INEQUALITY  AND  APPROXIMATING  EQUALITY  FORMS 

The  dual  programs  for  the  inequality  form  as  presented  in  Charnes- 
Cooper-Seif ord  [ 1 ]  are 
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(I) 


max 


v(6)  =  -5Tln 


s.t. 
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6TA2 
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+  y 


-  JT 

-  b 

=  b2T 


&,y  >  o 

and 

(ID  inf  £(,)  I  cTeAV  +  -  b’V  -  b2V 

o 

S.t.  -  z >  0 


To  obtain  the  dual  programs  for  our  approximating  (weighted)  equality 
form,  we  employ  the  procedure  in  [1].  Thus,  consider 

(1)  K(x,y,6,Y)  =  £  (c, exi  -  fi  x.)  +  £  (ec,e(yi/e)  -  y.yj 

iel1  1  11  iel2  1  1  1 


where  c.^  >  0  for  all  i  and  e  >0. 

Minimizing  with  respect  to  xi  and  y^  gives: 


(4) 

and 

(5) 


To  decouple,  as  in  [l],  we  obtain 
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when  we  choose 


(6)  x  =  A^z1  +  A2z2  ,  y  =  z2 


Thereby,  we  have  the  dual  problems 

max  v(S,y,c>  =  \  -e  Z2  Y, 

(III)  s.t. 


lei 


=  b1T 


rT.2  ^  T  _  .  2T 
o  A  +  y  =  b 

5  ,  y  >  Q 

,T,n  4  p  c/  \  4  1T  aV  +  A2z2  .IT  1  V2T  2  2T  (z2A) 

(IV)  inf  £(z)  =ce  -bz-bz+ec  ev,/ 

with  z  unconstrained 

(Note  that  the  c^,  ielg,  may  be  chosen  arbitrarily.) 

The  duality  theory  of  (III)  and  (IV)  is  precisely  that  of  the 
equality  case  in  [  1  ] ,  as  may  be  seen  by  making  the  change  of  variables  to 
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a  r 


Yi  =  eYi  *  ci  = 


ci  ,  iel1 
ec1,  iel£ 


Our  present  form  is,  however,  more  convenient  for  our  arguments. 
We  now  define 


/-  n  X  4  IT  A1 a1  +  A2Z2  .  IT  1  .  2T  2  2  .  n 

(7*1)  f(zj  -  ce  -bz-bz,z<0 

(n  o ^  4  „1T  A1  z1  +  A2Z2  .IT  1  ,2T  2  2T 

(7-2)  g(z,e)  =ce  -bz-bz+ec  e 
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We  will  then  have 
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Theorem  1 : 


For  some  e  ■  0,  inf  f(z)  exists  if  and  only  if  inf  fr(z.e)  exists. 
z2-  0  2 


Further, 


Proof: 


lim  inf  g(z,e)  =  inf  f(z) 
z  z2<0 


Consider  — .  It  may  be  written  as 


-  (c'k)’  e*V  ♦  a2*2  -  b2,  c^V')  •  l£l' 


A2,,2 

.  ,  A  3  (r  L  ) 

where  kr  =  yy- 

Z  i 


and  (c^k)1  ^  (c^,  •••»  cmkm^* 


For  z\  >  0  we  see  that  by  choosing  e  small  enough  we  can  make 

g(z,e)  an  increasing  function  in  that  direction.  Thus,  in  seeking  a  minimum 

p 

we  need  only  consider  z  <0.  For  notational  simplicity  in  the  following 

2 

and  "inf"  with  respect  to  z  shall  always  also  entail  z  <0. 

For  z^  K  0  and  all  e  >  0  we  have  ' 

(9)  f(z)  <  g(z,e)  <  f(z)  +  c. 

ier  1 

If  inr  f (z)  exists,  then,  by  (9),  inf(z)  <  g(z,e)  so  that  inf  g(z,e)  exists 

for  all  e  >  0.  Further,  inf  f ( z )  <  inf  g(z,e).  On  the  other  hand,  if 

inf  g(z,e)  exists,  then,  by  (9),  inf  g(z,e)  <  f(z)  +  c,  so  that  inf  f(z) 

ier  1 

exists.  Further,  too,  inf  g(z,e)  <  inf  f(z)  +  e  c.  . 

ier  1 


Hence 


(10.1) 


inf  f(z)  <  inf  g(z,e)  <  inf  f(z)  +  e 

and 


c 


i 


(10.2)  lim  inf  g(z,e)  =  inf  f(z) 
e-K) 

When  f(z)  has  a  minimum  at  z*,  again  from  (9)  we  can  conclude 
(11)  f(z*)  =  lim  g(z*,e) 

e-K)  Q.E 

O 

So  much  for  the  minimization  side  of  the  duality.  For  the  K  L 
side  we  have 


Theorem  2: 

The  maximum  in  problem  (I)  and,  as  e  -*■  0,  in  problem  (III)  is  the 


same. 


Proof: 


For  0  <  <  ec^,  we  have  -c^  <  y^ 


0,  whereas  for 


Yi  >  eCl> 
(12)  -e 


Therefore, 


e  /  .  c,  for  all 

itl 2  1 


>  0. 


Thus, 

(13)  v(6,y,e)  *£  v(<$)  +  c,  <  f(z)  +  c 

iel2  1  ier  1 


From  (13)  we  conclude 
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(14)  sup  v(<5,Y,e)  <  sup  v(6)  + 


Cl 


inf  f(z)  + 


•E  c, 

Ul2  1 


But  by  the  theory  of  [  1 J , 

sup  v(6,y,e)  =  max  v(6,y,o)  =  inf  g(z,e)  >  inf  f(z)  by  (10.1).  Hence 

(15)  inf  f(z)  <  max  v(6,y,e)  ^  sup  v(<5)  +  e  c,  <  inf  f(z)  +  e  c. 

ier  1  iel2 


Thus  letting  e  -*■  0,  we  have 


(16)  Inf  f(z)  <  lira  max  v(6,y,e)  <  sup  v(6)  <  inf  f(z) 

e-*0 


and 


(17)  lim  max  v(<5,y,e)  =  sup  v(6)  =  inf  f(z) 
e-K) 


Q.E.D. 
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